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ABSTRACT 

New results are presented, as part of the Hi-z Emission Line Survey (HiZELS), from the 
largest area survey to date (1.4 deg^) for Lya emitters (LAEs) at 2; '--^ 9. The survey, which 
is primarily targeting Ha emitters at 2 < 3, uses the Wide Field CAMera on the United 
Kingdom Infrared Telescope and a custom narrow-band filter in the J band and reaches a 
Lya luminosity limit of ^ lO^"^'^ erg s^^ over a co-moving volume of 1.12 x lO^Mpc'^ at 
z = 8.96 ± 0.06. Only 2 candidates were found out of 1517 line emitters and those were 
rejected as LAEs after follow-up observations. The limit on the space density of bright Lya 
emitters is improved by 3 orders of magnitude, consistent with suppression of the bright end 
of the Lya luminosity function beyond z ^ 6. Combined with upper limits from smaller but 
deeper surveys, this rules out some of the most extreme models for high-redshift Lya emitters. 
The potential contamination of future narrow-band Lya surveys at z > 7 by Galactic brown 
dwarf stars is also examined, leading to the conclusion that such contamination may well be 
significant for searches at 7.7 < z < 8.0, 9.1 < z < 9.5 and 11.7 < z < 12.2. 

Key words: galaxies: high-redshift, galaxies: luminosity function, cosmology: observations, 
galaxies: evolution. 



1 INTRODUCTION 

One of the most important questions in astronomy is "when did 
the first stars and galaxies form?". Observations of the most distant 
galaxies offer one of the greatest possible constraints on structure 
formation, allowing models of early galaxy formation and evolu- 
tion to be tested, refined or refuted. Over the last decade, consid- 
erable manpower and telescope time has been dedicated towards 
this goal: galaxies have now been identified out to redshift 2 ~ 7, 
just 750 Myr after the Big Bang l |Iye et al.|2006[ >, and recently a 
Gamma Ray Burst (GRB) has been detected even further away, 
at z ~ 8.3 ( [Tanvir et al.|2009] l; the sample of very-high redshift 
galaxies is growing rapidly. Making the additional step out to red- 
shifts of z ~ 9 is of the upmost importance because it not only 
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offers much tighter constraints on the first star formation or AGN 
activity of the Universe, but also allows the re-ionisation epoch of 
the Universe to be studied. As the mean fraction of neutral hydro- 
gen in the intergalactic medium increases, the Lya emission from 
these star forming galaxies will be strongly attenuated, with dra- 
matic consequences for the shape of the Lya luminosity function, 
although the precise details may depend upon the level of local 
ionization of the intergalactic medium by the star forming galaxies 
(e.g. [Haiman & Cen 2005 ). Little evolution is seen in the Lya lu- 
minosity function between z ~ 3 and 2 ~ 6, suggesting that the 
Universe was effectively fully ionized by 2 = 6 (e.g. |Malhotra &| 
|Rhoads|2004{|Ouchi et al.|2008^ , although some hints of evolution 
have been found at the bright end of the luminosity function beyond 
2 ~ 6 (e.g. |Kashikawa et al.|2006l|Ota et al.|2008| >. Still, the re- 
ionisation epoch is widely believed to occur around 2 ~ 9, with this 
being supported by several models and observations; the latest re- 
sults from the Cosmic Microwave Background (e.g. |Dunkley et af] 
i2009l|Komatsu et al.|2009j show that the bulk of the re-ionisation 
occurred at 2; = 10.9 ± 1.4 (la). 

Presently, there are three relatively effective methods for 
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searching for very distant galaxies: tiie broad-band drop-out tech- 
nique, "blind" spectroscopic searches and narrow-band imaging 
surveys. The widely-used drop-out technique (pioneered at z ~ 3 
by [Steidel erari[T996j requires very deep broad-band imaging, 
and can potentially identify z > 7 galaxies as z-band drop-outs 
(e.g. [Bouwens et"aLl|2008| [Richard et aLl[2008t . This method is 
efficient for identifying candidates, but requires detailed spectro- 
scopic follow-up to confirm the candidates, especially to rule out 
contributions from other populations with large z— J breaks, such 
as dusty or evolved z ~ 2 galaxies and ultra-cool galactic stars 
(e.g. |McLure et al.|2006[ l. Thus, while |Richard et al.|p008) identi- 
fied two z ~ 9 — 10 candidates by taking advantage of the lensing 
magnification of a high mass cluster, their spectroscopic follow-up 
was inconclusive with no emission lines detected. "Blind" spectro- 
scopic surveys can potentially provide spectra directly. They are 
always limited to very small areas, although |Stark et ari ( [2007| l tar- 
getted the critical lensing lines of clusters and were able to identify 
6 potential z ~ 9 objects. Finally, the narrow-band imaging tech- 
nique has the advantage of potentially probing very large volumes, 
but can only detect sources with strong emission lines, whilst it still 
depends on the Lyman-break technique to isolate very high-redshift 
emitters. 

Narrow-band Lya searches at slightly lower redshifts have 
been extremely successful in detecting and confirming emitters 
(e.g. |Hu et al!||1998^ , including the detection of the most distant 
(spectroscopically confirmed) Lya emitter to date at 2; = 6.96 ( |Iye| 
|et al. 2006). There have been attempts to detect Lya at z > 7, 
and particularly at z ~ 9 (e.g. |Willis & Courbin|2005||Cuby et al.| 
|2007[[Willis et al.|2008) , some taking advantage of cluster lensing 
magnifications; all such studies have been unsuccessful to date, but 
have only surveyed very small areas (a few tens of square arcmins 
at most). With the advent of wide-field near-IR detectors, however, 
it is now possible to increase the sky areas studied by over 2 to 3 or- 
ders of magnitude and reach the regime where one can realistically 
expect to detect 2: ~ 9 objects. This is a key aim of, for example, 
the narrow-band component of the Ultra VISTA Survey (c.f. |Nils^ 
|son et al.|2007| l. It is also an aim of HiZELS, the Hi-Z Emission 
Line Survey (c.f. [Geach et al.l|2008l [Sobral et al.]|2009) , that we 
are carrying out using the WFCAM instrument on the 3.8-m UK 
Infrared Telescope (UKIRT). HiZELS is using a set of existing and 
custom-made narrow-band filters in the J, H and K bands to de- 
tect emission line galaxies over ~ 5 square degrees of extragalactic 
sky; the narrow-band J filter (hereafter NBj) is sensitive to Lya at 
z = 8.96. 

In this work, an Ho = 70 km s^^ Mpc^^, ^Im = 0.3 and 
Qa = 0.7 cosmology is used; magnitudes are given in the Vega 
system. 



2 DATA AND SELECTION 



Deep narrow-band J (NBj ^ 21.6, 3a, Fum ~ 7.6 x 
10~^^ erg s^^ cm^^) imaging was obtained across 1.4 deg^ in the 
UKIRT Infrared Deep Sky Survey Ultra Deep Survey (UKIDSS 



UDS; [Lawrence et al. 


2007 1 and the Cosmological Evolution Sur- 


vey (COSMOS; Scovi 


le et al.|2007|[Koekemoer et al.[2007f fields. 



both of which have a remarkable set of deep multi-wavelength data 
available - this resulted in the selection of 15 17 potential line emit- 
ters. The NBj filter (A = 1.211 ± O.OlSpm) is sensitive to Lya 
emission at z = 8.96 ± 0.06 (assuming a top-hat filter shape), 
probing a co-moving volume of 1.12x lO*" Mpc^ - by far the largest 
probed by a narrow-band survey at these wavelengths. The reader 



is referred to [Sobral et al.[ ( (2009[ > - hereafter S09 - for details re- 
garding the observations, data reduction and the general selection 
of narrow-band emitters. Here the focus will be on identifying Lya 
emitters candidates within that data-set. 



2.1 Search for Candidates 

For a source to be considered a candidate z m 9 Lya emitter it 
is required to: i) be selected as a narrow-band emitter in S09 (this 
required it to be clearly detected in NBj (a > 3) with a J-NBj 
colour excess significance of E > 2.5 and observed equivalent 
width EW> 50 A - see S09 for details in which it is shown that 
these criteria are very robust); ii) have at least one other detection 
> 3(7 in the near-infrared; iii) be visually believable in NBj and the 
other band(s), avoiding noisy areas; and iv) be undetected (< 3a 
and direct visual analysis) in the available visible band imaging 
{B,V,r,i,z) - SUBARU and ACS/HST). 

The sample presented in S09 was used to search for potential 
Lya emitters at z ~ 9. However, the investigation was also ex- 
tended to a slightly larger area in the UKIDSS UDS field to include 
areas where deep SUBARU and near-infrared imaging data were 
available - this corresponds to re-including areas which were con- 
servatively masked for SED fitting purposes in S09, and increases 
the total area probed to 1.4 deg^. 



2.2 Candidates, testing and follow-up observations 

No candidates were found in the UKIDSS UDS field, with all emit- 
ters that passed tests i) to iii) being clearly detected in z-band imag- 
ing. In COSMOS, however, 2 candidates were found that satisfied 
all criteria. Both sources are absent in all optical bands down to the 
3a level (e.g. I — 28.1 mag, z = 25.8 mag). They are both de- 
tected in NBj and J, with Cand I having NBj = 20.8 mag {5a) 
and J = 21.5 mag (9a) and a drop z — J > 4.2 mag, while Cand2 
presents NBj — 20.8 mag (5a) and J = 22.1 mag (6a) and a drop 
z — J > 3.6 mag. They are both undetected in all other infra-red 
bands. 

These two sources were then subjected to a series of further 
tests and follow-up observations. Splitting the data into subsets 
confirmed the detections across observations conducted on differ- 
ent nights (timescale from one day up to one month), with no 
evidence for variability or proper motion; they were also clearly 
excluded as potential cross-talk artifacts. Candl was foUowed-up 
spectroscopically using the CGS4 instrument on UKIRT in Jan- 
uary 2009 ~ these data failed to confirm an emission line. Both 
candidates were then re-observed using WFCAM (further J imag- 
ing in February 2009), resulting in the non-detection of both can- 
didates. Also, J imaging from the COSMOS public archive (which 
has become publicly available very recently) fails to detect the can- 
didates. It is therefore clear that these sources are not Lya emitters 
at z ~ 9. Further investigation shows that the sources are likely to 
be artifacts; they are located at almost the same pixel position on 
WFCAM4 in 2 different paw-prints (later investigation showed that 
these were also found at similar pixel positions in the remaining 2 
paw-prints, although at a lower level, and thus were not selected). 
They seem to be caused by an unfortunate coincidence of a set of 
slightly hot pixels (not sufficient to be flagged as bad pixels) which, 
combined with the ditter pattern, produced a few a excess at one 
location on the combined image. No other artifacts like these were 
found either in UDS (which used a different dither pattern) or in 
other cameras for COSMOS data. 
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Figure 1. Comparison between the measured Lya luminosity function at 
^ ~ 3 (dott ed lines;|Gronwall et al.|2007||Ouchi et al.|2008t with data from 
2 ~ 6 - 7 jKashikawa et al.|2006||Shimasaku et al.|2006||Ota et al.|2008t . 
Other typically smaller 2^6 surveys give consistent results within the 
error bars. No evidence of significant evolution is found, especially when 
accounting for cosmic variance. Limits for the z ~ 9 LF from |Willis &| 
|Courbin|(2005) , |Cuby et~ar 1 2007 1 and Wilhs et al. (20081 are also pre- 
sented, together with the one presented in this Letter, which is inconsistent 
with a strong evolution in L* up to 2 ~ 9. 



3 RESULTS & DISCUSSION 

3.1 Lya luminosity function at z ~ 9 

After conducting the widest survey of bright (L > 7.6 x 
10*'^ erg s^^) Lya emitters at z ~ 9, probing a co-moving vol- 
ume of 1.12x10® Mpc^ (L4 deg^), only 2 sources passed the se- 
lection criteria and even those were ruled out after follow-up obser- 
vations. This result allows the tightest constraint on the bright end 
of the 2 ~ 9 Lya luminosity function, as previous surveys l |Willis| 
|& Courbin|2005| |Cuby et a.l.|2"007) [Willis et al.|2008l l have only 
covered very small areas (a factor ~ 1000 smaller). However, those 
surveys have gone significantly deeper (up to a factor of ~100). 
Thus, by combining all the results from the literature, the luminos- 
ity function of LAEs at z ~ 9 can be constrained across a wide 
range of luminosities: 10''^ <L < lO^'^ergs^^ Fi gure[I]p resents 
the constraints from |Willis & Courbin| ( |2005) , |Cuby etaLjpOOTl l, 
|Willis et aLl l |2008t and from this work, indicating the inverse of the 
volume selection function for each survey. These are compared to 
the measured Lya luminosity functions from 2 ~ 3 to z ~ 7 from 
recent studies. Although the samples of Lya emitters may suffer 
from significant biases due to the selection, cosmic variance and 
possible contamination. Figure [T| reveals that there is little evolu- 
tion in the bright end of the luminosity function between z ~ 3 
and z ~ 5.7. However, those bright emitters seem to become much 
rarer at z = 6.5 ( [Kashikawa et al.|2006| , indicating that L* is not 
increasing from z ~ 6 onwards. The results presented in this Letter 
are also consistent with no evolution in L* (Alog(L*)<0.5) from 
z = 5.7 to z ~ 9. 



3.2 Comparison with models and future surveys 

Several authors have tried to predict the Lya luminosity func- 
tion at z ~ 9, either by extrapolating the luminosity function of 
these emitters from lower redshift, or by using numerical or semi- 
analytical models ( [Thonmies & Meisenheimer||2005[ |Le Dellioul 
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Figure 2. The observational limits on the 2 ~ 9 Ly« luminosity func- 
tion compared to different model predictions and proposed future surveys. 
The most recent models agree well with the data limits, and only the most 
extreme ones can be ruled out. Also, according to these models, the Ul- 
traVISTA survey will have a clear chance of detecting a few of these Lya 
emitters, while ZEN3 may get a detection and will at least be able to rule 
out more models. 



let al.||2006| [Nilsson et aLl|2007| >. In this Letter, different models 
are compared with the observational constraints: semi-analytical 
models, observational extrapolations and phenomenological mod- 
els. The semi-analytical models discussed here are obtained from 
GALFORM (Baug hetal.|20 05 ) - these are based on ACDM, having 
been successful in reproducing a wide range of galaxy properties at 
different redshifts, including Lya emitters up to z ~ 6 ( |Le Dellioul 
|et al.|2006t . GALFORM computes the build-up of dark matter halos 
by merging and the assembly of baryonic mass of galaxies and the 
semi-analytical approach allows the study of properties of the Lya 
emission - the reader is referred to Baugh et al.| ( |2005^ , |Le Del-| 
|liou et al.|p006l > and |Orsi et aLlp008| > for" more details on these. 
The observational approach, as in Nilsson et al.| ( [2007) , extrapo- 
lates the Schechter function parameters based on those obtained in 
the 3.1 < z < 6.5 redshift range. In practice, this results in little 
L* evolution but a significant negative (j>* evolution. Finally, the 
phenomenological approach in |Thommes & MeisenheimerH2005] > 
assumes that Lya emitters at high redshift are spheroids seen dic- 
ing their formation phase. These models are normalised to give the 
observed mass function of spheroids in the local Universe, and are 
combined with a phenomenological function that provides the dis- 
tribution of spheroid formation events in mass and redshift. Each 
galaxy is assumed to be visible as a Lya emitter during a starburst 
phase of fixed duration that occurs at a specific redshift, drawn from 
a broad distribution. The reader is referred to lThommes & Meisen-I 
|heimer|p005t for details. 

Figure |2] presents predictions from GALFORM ( |Le Delliou| 
|et al.[ 2006 ), the observational luminosity function extrapolation 
fromlNilsson et al. (20071 and updated phenomenological predic- 
tions |Thommes & Meisenheinier|2005jl assuming peak redshifts of 
Zmax = 3.4 and Zmax = 5.0. |Le Delliou et ar]p006[ l found that 
their model required an escape fraction of fcsc = 0.02 to fit the ob- 
served Lya luminosity function at 3 < 2 < 6.5, but they also pre- 
sented predictions for fesc = 0.2 to illustrate how the results might 
change if the escape fraction increased at very high redshift; we 
therefore show both predictions in Figure|2] While most predictions 
are consistent with the current limits, GALFORM models with high 
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escape fractions are marginally rejected both at faint and bright lev- 
els. Earlier phenomenological models (e.g. the Zmax = 10 model 
of [Thommes & Meisenheimer||2005| not shown in Figure [2]l are 
also clearly rejected by our results. 

These results also show, observationally, that bright L > 
10*^'* ergs~^ Lya emitters are very rare. Although the area cov- 
erage is absolutely important, a depth-l-area combination is likely 
to be the best approach for gathering the first sample of these 
very high-redshift galaxies. In fact, that is the strategy of the 
narrow-band component of the Ultra VISTA survey (c.f. |Nilsson| 
|et al.|[2007^ , using the VISTA telescope, which will map 0.9 
deg^ of the COSMOS field to a planned 5a flux limit of 4 x 
10~^* ergs~^ cm~^; this corresponds to luminosity limit of L = 
10*^'^^ erg s~^ and a surveyed volume of 5.41 x 10^ Mpc^ (see Fig- 
ure|2]l ai z = 8.8. This combination lies below all current predic- 
tions for the z ~ 9 Lya LF and the survey is expected to detect 2-20 
Lya emitters al z — 8.8±0.1. On the other hand, the ZEN3 survey 
(c.f. |Wims et al.|2008| Hibon et al. 2009, submitted to A&A) will 
also try to get a compromise between area and depth by taking ad- 
vantage of the Canada-France-Hawaii Telescope (CFHT) and their 
near-infrared large area camera (WIRCam); that survey will not go 
as wide or as deep, and whilst it will provide significantly better 
constraints, it is not clear (as can be seen in Figure |2| whether or 
not it will be successful in detecting any Lya emitter. 

3.3 High redshift Lya searches and cool galactic stars 

It has become widely realised in recent years that broad-band 
searches for z > 6 galaxies using the Lyman-break technique may 
suffer from significant contamination by cool Galactic L, T, and 
possibly Y-dwarf stars (e.g. [McLure et al.|20()6^ . These low-mass 
brown dwarfs display extremely red z — J colours reaching as high 
as z — J ~ 4 (e.g. |Bumingham et al.|2008] >, coupled with relatively 
flat J — K colours. Such colours can mimic very closely those ex- 
pected of a 2 > 6 star forming galaxy with a strong Lyman-break. 

It may be thought that narrow-band Lya searches are im- 
mune to this contamination, since the initial emission-line galaxy 
selection relies on an excess flux observed in a narrow-band fil- 
ter relative (usually) to a broad-band filter; only after that is the 
Lyman-break technique used to pick out the high-redshift Lya can- 
didates from amongst the emission-line objects. However, the near- 
infrared continuum spectra of low mass brown dwarfs show consid- 
erable structure due to broad molecular absorption features (espe- 
cially methane and ammonia; e.g. Legg ett et al.||2007) , as shown 
in the top panel of Figure [5] The lower panel of Figure [5] shows 
very clearly that T-dwarfs can easily produce a positive broad- 
band minus narrow-band (BB-NB) colour if the narrow-band fil- 
ter is located within one of the spectral peaks (note that this is 
much less of an issue for surveys which difference two closely- 
located narrow-band filters). Lya narrow-band surveys in the red- 
shift ranges 7.7 < z < 8.0, 9.1 < z < 9.5 and 11.7 < z < 12.2 
may therefore be prone to contamination by cool Galactic stars 
- this includes the z — 7.7 and z — 9.4 atmospheric windows 
for narrow-band searches of Lya emitters. Narrow-band surveys at 
redshifts z < 7.7, or between 8.0 < z < 9.1 - which includes 
both HiZELS (z = 8.96) and the narrow-band component of the 
Ulti-a VISTA Survey (z = 8.8; e.g. Nilsson et al 2007) - will be 
free of such contamination. Indeed, such surveys could potentially 
select very cool T-dwarf stars via a narrow-band deficit due to the 
strong methane absorption feature at these wavelengths. 

With this result in mind, a T-dwarf search was conducted 
among narrow-band deficit sources in S09. These deficit sources 
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Figure 3. Top panel: the near-infrared spectra of TO, T3, T6 and T9 dwarf 
stars (TO - lighter, T9 - darker, from |Bumingham et al.|2008|l ) compared 
to near-IR broad band filter profiles. Lower panel: the consequences for 
measured broad-band minus narrow-band (BB-NB) colours, clearly demon- 
strating the redshifts/wavelengths at which searches for Lya emitters can be 
significantly contaminated by these very cool stars. For 7.7 < 2 < 8.0 and 
9.1 < z < 9.5 searches, these stars can easily mimic Lya emitters, with 
strong Y-x or J-z breaks and significant positive BB-NB colours. Searches 
at higher redshift 11.6 < 2 < 12.2 in the H band can detect T9s with 
BB-NB~1.5, although the lack of strong H-J or H-Y breaks will make it 
easier to distinguish T-dwarfs from Lya emitters. 



were selected using equivalent criteria as for emitters (with a 
change in sign). None of the deficit sources has z(AB)— J > 3, 
as expected for T-dwarfs (e.g. Leggett et al. 2007, Burningham et 
al. 2008), and even a selection imposing z(AB)— J > 2 results 
in a sample of only 9 galaxies which are all very well SED-fitted 
as galaxies with Zphoto ~ 1-4 — 1.5. These sources also present 
slightly higher J and H fluxes when compared to the best SED fit, 
but this can be explained by the H/3 and [Olll] contributing to the J 
band and Ha to the H band, which also explains the NBj deficit. 
No T-dwarf candidate was found in our survey. 



4 SUMMARY 

• Deep narrow-band imaging in the J band (A = 1.211 ± 
0.015/im) has been used to search for bright Lya emitters at 
z — 8.96 over an area of 1.4 deg^. No Lya emitter was found 
brighter than L 7.6 x lO"^ ergs~\ 

• The Lya luminosity function constraints at 2: ~ 9 have been 
improved for lO**^ < L < lO*'' ergs~^ emitters. The results rule 
out significant positive evolution of the Lya LF beyond z ^ 6; they 
are in line with recent semi-analytic & phenomenological model 
predictions, rejecting some extreme models. 

• It has been shown that for narrow-band searches, T-dwarfs can 
mimic Lya emitters at 7.7 < z < 8.0, 9.1 < z < 9.5 and 11.7 < 
z < 12.2; they will not contaminate the future Ultra VISTA narrow- 
band survey (and can even be identified via a narrow-band deficit), 
but they may contaminate narrow-band Lya searches within the 
z = 7.7 and z — 9.4 atmospheric windows. 
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